Method And Apparatus For Identifying Different Routing Paths Between Networks

ABSTRACT

A method and apparatus is disclosed for analyzing IP data flows for the determination of an alternate routing path for network traffic between a known first network and a destination within an unknown second network on the Internet.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/694,869, filed Apr. 23, 2015, which is hereby incorporated herein byreference in its entirety for all purposes. U.S. application Ser. No.14/694,869 claims the benefit of U.S. Provisional Application Ser. No.62/007,565, filed Jun. 4, 2014, which is hereby incorporated herein byreference in its entirety for all purposes.

TECHNICAL FIELD

The present disclosure relates to computer networking and communicationsnetworks. More particularly, the present disclosure relates to wide areacomputer networking and wide area communications networks.

BACKGROUND

Network operators interconnect their autonomous system networks (ASNs)through some mix of Internet transit relationships, where a fee is paidfor a network to carry traffic, and peering relationships, where feesare substantially avoided, forming the Internet. Turning to FIG. 1A,Internet traffic can flow by various methods. Traffic originating at asource autonomous network or first network 100 can access resources of adestination within a destination autonomous system network or a secondnetwork 118 via several methods.

First, the first network 100 can send traffic originating from itssource router 102 via a network communications link 104 to a firsttransit provider autonomous systems network (“first transit provider”)106. The first transit provider 106 would then direct said traffic via aseries of internal or external routers 108, 112 that are connected toone another via one or more network communications links 110 at thediscretion of the first transit provider 106. The traffic is then sentdirectly to the destination router 116 of the second network 118 via anetwork communications link 114. Furthermore and at the discretion ofthe first transit provider 106, traffic could also be directed vianetwork communications link 136 to one or more additional transitproviders in series 138, 140, 142, 144 via their respective networkcommunication links 146, 148, 150, in which such traffic is controlledat the discretion of the first transit provider 106 before reaching thedestination router 116 within the second network 118 via networkcommunications link 152.

Second, the first network 100 can direct traffic through an intermediateInternet Exchange Point (IXP). An IXP is a physical infrastructurethrough which Internet Service Providers (ISPs), Content DeliveryNetworks (CDNs) and Enterprise networks exchange traffic between theirautonomous system networks. The primary purpose of the IXP is to allowautonomous system networks to interconnect directly, via the IXP, ratherthan through one or more third-party networks. The advantages ofdirecting traffic through an intermediate IXP interconnection arenumerous and include improvements in cost savings, latency, bandwidthand routing control.

A typical IXP includes one or more Ethernet-based local area network(LAN) switches housed in a single location or interconnected acrossmultiple locations. The IXP operates in a layer-2 configuration andutilizes an IP subnet for the connection of participating autonomoussystem networks. While these very design choices have made thetraditional IXPs useful, the IXPs are centralized in their uses orapplications.

The second method of accessing resources of the second network 118 isalso shown in FIG. 1A. The known first network 100 can direct trafficfrom the source router 102 over a network communications link 120 to anIXP 122. The IXP 122 directs the traffic through the IXP switch fabric124 to the network communications link 126 and to the destination router116 of the second network 118.

Finally, the known first network 100 can establish a privateinterconnection network communications link 134 directly with the secondnetwork 118 for the purpose of sending traffic between them, as alsoshown in FIG. 1A.

SUMMARY

In a first aspect, the present disclosure may provide acomputer-implemented method for identifying alternate routing paths fornetwork traffic between a known first network and a destination withinan unknown second network on the Internet. The network traffic betweenthe first network and the destination within the unknown second networkmay have an initial routing path that crosses at least oneinterconnecting transit network. The method may comprise obtaining, at aprocessor of a computer, destination IP address information for thedestination within the unknown second network. The processor may comparethe destination IP address information to an Internet database todetermine an identity of the second network. The processor may further,using the identity of the second network, identify a second routing pathcomprising a peer connection between the first network and the secondnetwork. The peer connection and the second routing path may differ fromthe initial routing path and may bypass the at least one interconnectingtransit network.

The peer connection may be across a distributed Internet exchangeplatform (DIXP), across an Internet exchange point (IXP), or a directconnection. Additionally, the method may further comprise obtaining, atthe processor, source IP address information. In such examples, theprocessor using the identity of the second network to identify a peerconnection between the first network and the second network may furtheralso use the source IP address information to identify the peerconnection.

In some examples, the processor using the identity of the second networkto identify the peer connection between the first network and the secondnetwork bypassing at least one interconnecting transit network may use anetwork map of interconnections among networks on the Internet toidentify the peer connection. In certain examples, the method mayfurther comprise establishing the peer connection between the firstnetwork and the second network. Also, in certain examples, the methodmay further comprise obtaining flow data representing traffic flowpatterns between the first network and the second network across atleast one interconnecting transit network and extracting the destinationIP address information from the flow data. Other aspects and features ofthe present disclosure will become apparent to those ordinarily skilledin the art upon review of the following description of specificembodiments in conjunction with the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the invention will become more apparent fromthe following description in which reference is made to the appendeddrawings wherein:

FIG. 1A illustrates methods known in the prior art of trafficoriginating at a first known network accessing resources of adestination within a second network;

FIG. 1B illustrates an exemplary method for Internet traffic flow inwhich the first known network directs traffic through a distributedinternet exchange platform (DIXP);

FIG. 2 illustrates the details of an exemplary DIXP of FIG. 1B;

FIG. 3 is a flow chart illustrating an exemplary computer-implementedmethod for identifying alternate routing paths for network trafficbetween a known first network and a destination within an unknown secondnetwork on the Internet;

FIG. 4 is a flow chart of an exemplary arrangement for collectingnetwork analytics in support of the exemplary computer-implementedmethod of FIG. 3; and

FIG. 5 is a block diagram of a computer system in respect of which theexemplary methods herein described may be implemented.

DETAILED DESCRIPTION

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the figures herein,can be arranged and designed in a wide variety of differentconfigurations. Thus, the following more detailed description, asrepresented in the figures, is not intended to be limiting in scope, asclaimed, but is merely representative of certain examples. The presentlydescribed examples will be best understood by reference to the drawings,wherein like parts are designated by like numerals throughout.

Generally, the present disclosure provides a method and system fordetermining the analytics around routing and flow metrics to assist indetermining whether to join an Internet exchange platform (IXP), adistributed Internet exchange platform (DIXP), or engage in simpleprivate interconnection as an alternative to a traffic flow path thatcrosses one or more interconnecting transit networks. It is to beappreciated that as used herein, the term “interconnecting transitnetwork” refers to transit provider autonomous systems networks, orASNs, as described above, and does not include an IXP, a DIXP or simpleprivate interconnection. A DIXP is a service provider network directlyinterconnecting at least some of the available IXPs globally (Network ofIXPs) and consists of a set of related services as described in WIPOpatent publication number WO2014/059550, the teachings of which arehereby incorporated by reference.

Identifying and analyzing the dominant traffic flow patterns enablesnetwork operators to identify other interconnected networks for Internettransit and peering. Collecting network traffic data can be accomplishedby periodically collecting flow data from network routers, switches orother computer network devices (“Network Device”), either by using datasampled and delivered by Netflow (RFC 3954), sflow (RFC 3176), cflowd(RFC 3954), jflow (a proprietary protocol supported by Juniper Networks)or other similar protocol (each referred to individually as a “FlowProtocol”), or by analyzing the raw traffic data delivered over suchnetwork device port by a port mirror or network tap to provideinformation including the source and destination Internet Protocol (IP)address tuples and associated traffic flow, including routing direction,next hop on the route and volume (“Flow Data”).

FIG. 1B shows an exemplary method for Internet traffic flow in whichtraffic flows from a first network 100 through a physical infrastructure130 operable to provide a peer connection between the first network 100,which may be an autonomous system network (ASN), and a second network,which may also be an ASN. Examples of such physical infrastructure 130may include, without limitation, a DIXP, an IXP, and/or a privateconnection. The peer connection may be provided over a routing paththrough the physical infrastructure that differs from an initial routingpath between the first network and the second network. In such examples,the initial path may have traversed one or more interconnecting transitnetworks 106, 138, 140, 142, 144, as in the first scenario describedabove with respect to FIG. 1A.

With reference to FIG. 1B and FIG. 2, the first network 100 can sendtraffic through network communications link 128 into the DIXP 200. TheDIXP can then send the traffic through the switch fabric amongst itsgeographically diverse switches 202, 216, 218, 220 via networkcommunication links 204, 206, 208, 210, 212, 214. The DIXP can directtraffic via the geographically diverse switches 202, 216, 218, 220 usingvarious algorithms, standards and/or protocols, including but notlimited to: Shortest Path Bridging (SPB), as defined by IEEE 802.1aq;Transparent Interconnection of Lots of Links (TRILL); Spanning TreeProtocol (STP), as defined by IEEE 802.1D or other such layer 2protocols. The traffic is sent to the destination router 116 within thesecond network 118 via the network communications link 132.

Reference is now made to FIG. 3, which is a flow chart illustrating anexemplary computer-implemented method 300 for identifying alternaterouting paths for network traffic between a known first network and adestination within an unknown or unidentified second network on theInternet. The flowcharts in FIG. 3 and FIG. 4 illustrate thearchitecture, functionality, and/or operation of possibleimplementations of systems, methods, and computer program productsaccording to examples. In this regard, each block in the flowcharts mayrepresent a module, segment, or portion of code, which comprises one ormore executable instructions for implementing the specified logicalfunction(s). It will also be noted that each block of the flowchartillustrations, and combinations of blocks in the flowchartillustrations, may be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

Where computer program instructions are involved, these instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block or blocks. These computer programinstructions may also be stored in a computer readable medium that maydirect a computer to function in a particular manner, such that theinstructions stored in the computer-readable medium produce an articleof manufacture including instruction means which implement thefunction/act specified in the flowchart and/or block or blocks. Thecomputer program may also be loaded onto a computer to cause a series ofoperation steps to be performed on the computer or other programmableapparatus to produce a computer implemented process for thefunctions/acts specified in the flowchart and/or block or blocks.

It should also be noted that, in some alternative implementations, thefunctions noted in the blocks may occur out of the order noted. Incertain embodiments, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved. Alternatively, certain steps or functions may be omitted.

The method 300 depicted in FIG. 3 is a computer-implemented methodcarried out by one or more computer systems, potentially by one or moremodules residing at the one or more computer systems and/or elsewhere ina system in which the computer system(s) play a role, in particular bythe processor(s) thereof, with the processors controlling ancillaryhardware. As used herein, the term “processor” may include multipleprocessing devices operating in cooperation as a processing unit.Modules may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.), or an embodiment combining software and hardwareaspects. Furthermore, aspects of the presently discussed subject mattermay take the form of a computer program product embodied in any tangiblemedium of expression having computer-usable program code.

With respect to software aspects, any combination of one or morecomputer-usable or computer-readable media may be utilized. For example,a computer-readable medium may include one or more of a portablecomputer diskette, a hard disk, a Random Access Memory (RAM) device, aRead-Only Memory (ROM) device, an Erasable Programmable Read-Only Memory(EPROM or Flash memory) device, a portable Compact Disc Read-Only Memory(CDROM), an optical storage device, and a magnetic storage device. Inselected embodiments, a computer-readable medium may comprise anynon-transitory medium that may contain, store, communicate, propagate,or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

Computer program code for carrying out operations of the presentinvention may be written in any combination of one or more programminglanguages, including an object-oriented programming language such asC++, and conventional procedural programming languages, such as the “C”programming language, or similar programming languages. Aspects of amodule that are implemented with software may be executed on amicro-processor, Central Processing Unit (CPU) and/or the like. Anyhardware aspects of the module may be implemented to interact withsoftware aspects.

In the method 300 shown in FIG. 3, an initial routing path for thenetwork traffic between the first network 100 and the destination withinthe unknown second network 118 crosses at least one interconnectingtransit network (e.g. transit provider autonomous systems networks 106,138, 140, 142, 144 in FIGS. 1A and 1B). However, there may be alternaterouting paths which bypass the interconnecting transit network(s) byusing a peer connection, such as one or more IXPs, one or more DIXPs orone or more private interconnections. Such alternate routing paths maybe advantageous relative to the initial routing path. For example, theuse of a peer connection instead of one or more interconnecting transitnetworks can reduce latency and jitter, and may avoid the transit feesassociated with the interconnecting transit network(s). However, becausethe second network 118 (i.e. the network that contains the destination)is unknown, alternate routing paths via peer connection may not beapparent.

At step 302, the method 300 may obtain, at a processor of a computer,destination IP address information for the destination within theunknown second network. It may do so by obtaining flow data representingtraffic flow patterns between the first network and the second networkacross the interconnecting transit network(s), and extracting thedestination IP address information for the destination within theunknown second network from the flow data, thereby obtaining thedestination IP address at step 304. In some embodiments, thisfunctionality may be performed by a collection module residing at one ormore computers.

At optional step 304A, with arrows shown in dashed lines, the method 300may also extract source IP address information from the flow data. Insome embodiments, this functionality may be performed by a processingmodule residing at one or more computers. Steps 302 and 304 (and 304A,when present) may be performed by the same computer system, or a firstcomputer system may extract the destination IP address information (andsource IP address information) from the flow data at step 302 and passit to a second computer system, which receives it at step 304 (and304A).

At step 306, the processor compares the destination IP addressinformation to one or more databases, such as an Internet database, todetermine the identity of the unknown second network. Such databases maybe operable to maintain destination internet protocol (IP) addressescorrelated to a set of networks and/or routing information for the setof networks. This functionality may also be performed by a processingmodule residing at one or more computers.

Once the identity of the unidentified second network is determined, themethod 300 at step 308 may use the identity of the second network toidentify a peer connection, which may also be determined by theprocessing module, between the first network and the second network, forexample by using a network map of interconnections among networks on theInternet. The peer connection identified at step 308 may bypass theinterconnecting transit network(s), and may be, for example, a peerconnection across a DIXP, a peer connection across an IXP, or a directconnection. Where step 304A is present, step 308 may also use the sourceIP address information to identify the peer connection, and may useadditional information as well.

In examples including the processing module, the processing module maybe communicatively coupled to the database(s) and may have a processor,memory unit, and executable code. The processing module may be operableto receive a destination IP address for network traffic and compare thedestination IP address to information in the database(s) to determine anidentity of the second network. Additionally, the processing module maydetermine the routing path to the second network based on the identityof the second network and the routing information.

At optional step 310, shown in dashed lines, the method 300 mayestablish, potentially by an implementation module, the peer connectionbetween the first network and the second network. Alternatively, thepeer connection may be established manually. In examples including theimplementation module, the implementation module may reside with thephysical infrastructure over which the peer connection is implementedand/or the one or more computers. Additionally, the implementationmodule may be communicatively coupled to the processing module and/orthe physical infrastructure. The implementation module may be operableto receive the routing path from the processing module and to establishthe peer connection implementing the routing path between the firstnetwork and the second network via the physical infrastructure.

FIG. 4 is a logical flow diagram of an exemplary arrangement forcollecting network analytics to support the method 300 described above;for convenience the exemplary arrangement is referred to herein as a“Network Flow Finder System”. In one presently preferred embodiment, theNetwork Flow Finder System comprises the source equipment 400 (forexample, a router or a switch), a data collector 418, or collectionmodule 418, a manual aggregate capture application 410, a database 440containing information related to the operations of the internet(Internet database 440), including registry and geographical informationand a data processor 426, or processing module 426, resulting in variousoutputs, including a report for manual review as well as automatedtriggers both based on results as well as statistical analysis of thosestatistics over time to notify third party systems of the changes.

In more detail, as shown in FIG. 4, the process may begin with datacollection from one or more network devices 400. For example, two ormore methods may be used to obtain data from the network device: capturevia flow protocol 404 or via network tap/port mirror 406. In eithercase, the network device 400 can either provide raw data or aggregatedinformation and sampled data, including flow protocol datagrams toeither the data collector/collection module 418 and/or the manualcapture to aggregate application 410.

Furthermore, as shown in FIG. 4, the manual capture to aggregateapplication 410 allows users to obtain traffic information withouthaving a direct connection to the data collector/collection module 418or the proceeding processes covered in the disclosure. For privacy andsecurity purposes, if the user elects to utilize this manual capture toaggregate application 410, the application may remove identifyinginformation of the data transferred resulting in an identifier for theautonomous system network, the percentage of traffic transferred to thatnetwork and other relevant aggregate counters. Depending on the level ofinformation removed, it may effectively remove proprietary orconfidential information.

When data arrives in the raw analysis to aggregate process 412 as rawdata from the network tap/port mirror 406, the data may be analyzed,aggregated progressively, and passed to an aggregate capture to fileprocess 414. From there, the aggregate capture to file process 414 maysave the aggregate information as a file on the machine on which theapplication is running as a tabular data format in either human readableformat, including but not limited to comma/tab separate values(CSV/TSV), extensible markup language (XML), javascript object notation(JSON), as well as computer readable format, including but not limitedto binary data in any type of encoding or format or proprietary dataformats. At this point the user can elect to upload 416 said data to thedata collector 418 for further processing. In some example, thecollection module may further comprise an application programminginterface (API) 444. Such an API 444 may be configured to enable flowdata, such as in a file, to be uploaded to the collection module from auser of the system.

Now referring to the data collector/collection module 418 in FIG. 4, itis shown that the data collector/collection module 418 can receive dataabout traffic flowing through the network equipment both via theaggregate and sample data via flow protocol datagrams 408 as well as viaraw network traffic using a network tap/port mirror 406. In the case ofreceiving raw data via the network tap/port mirror 406, the traffic maybe forwarded to the data collector 418 then on to the traffic analyzer422 that captures the data in real time to create flow protocol-likedatagrams and passes them along to the aggregator 420. In the case ofaggregate and sample data via network datagrams 408, the datagrams maybe forwarded to the data collector/collection module 418 and onto theaggregator 420 to combine the data. Once the aggregator 420 hascompleted its capture of data, it prepares the aggregate data 424 to besent to the data processor/processing module 426 either through internalcommunication or via web services call to the data processor/processingmodule 426.

Furthermore, the data processor/processing module 426 in FIG. 4 may bethe analysis process that assimilates the aggregate data 428 andlinks/relates 432 the aggregate data to the Internet database 440 toidentify key potential autonomous system networks that would yieldperformance, cost savings or other metric-based goals. Aggregate datapassed from the data collector/collection module 418 to the dataprocessor/processing module 426 may be processed with the option ofstoring the aggregate data along with metadata 430 within the Internetdatabase 440 for purposes including but not limited to improving futureanalysis, monitoring progress over time 458 and utilizing artificialintelligence (AI) algorithms, which may be applied by an AI module 460,to improve predictability.

In examples where the processing module 426 further comprises an AImodule 460, the AI module 460 may be communicatively coupled to thecollection module 418 and/or the database(s) 440. Such an AI module 460may be operable to receive flow data from the collection module.Additionally, the AI module 460 may generate the routing information forthe set of networks by applying one or more AI algorithms to the flowdata and provision the routing information to the database 440.

The aggregate data may be linked/related 432 to the database 440, whichmay include a variety of databases, including the various regionalInternet registry databases 446, the IP address to geographicalinformation databases 448, IXPs, ASN-IXP membership databases 450 and/orother relevant databases 452. These databases 440 may be accessed from alocal network database 442. Third party databases may be accessible viaa web service interface or application programming interface (API) 444.The aggregate data and or peer connection in the routing information maythen be ranked based on metrics, such as by a ranking module 434,including but not limited to geographical proximity, traffic volumes,latency or jitter and from/to the source or first network and thedestination or second network. The results may be processed 436 with theoption of saving them for future use 438 as described above. By doingso, this process may identify potential destination ASNs for directinterconnection via the DIXP and the amount of traffic that could bedirectly exchanged across the DIXP with said ASNs.

The result from the data processor/processing module 426, as shown inFIG. 4, may include computed suggested peer connections 437, provided bya suggestion module 437. The benefits of the computed, suggested peerconnections 437 are outlined above. The information in the computedsuggested peer connections 437 can be delivered in a variety of waysincluding a report for manual review 454, a signal/trigger to otherapplications via methods including, but not limited to, local script,web services, local & remote APIs and remote procedure calls 456, orcompared over time 458 to determine patterns over time using artificialintelligence algorithms 460 to signal/trigger other applications 456 asdescribed above. The report and trigger information provides the levelof abstraction needed to easily identify where services and applicationsare currently delivered for its customers and end users, and how theymight be better delivered via specific destination ASNs across the DIXP.Prospective DIXP customers can then determine the desired mix ofavailable destination ASNs for direct (physical or virtual)interconnection, Internet transit providers and interconnectionlocations available across the DIXP, providing a quantifiable assessmentof the incremental value derived by using the DIXP.

An illustrative computer system in respect of which the methods hereindescribed may be implemented is presented as a block diagram in FIG. 5.The illustrative computer system is denoted generally by referencenumeral 500 and may include a display 502, input devices in the form ofkeyboard 504A and/or pointing device 504B, computer 506 and/or externaldevices 508. While pointing device 504B is depicted as a mouse, it willbe appreciated that other types of pointing device may also be used.

The computer 506 may contain one or more processors or microprocessors,such as a central processing unit (CPU) 510. The CPU 510 may performsarithmetic calculations and control functions to execute software storedin an internal memory 512, preferably random access memory (RAM) and/orread only memory (ROM), and possibly additional memory 514. Theadditional memory 514 may include, for example, mass memory storage,hard disk drives, optical disk drives (including CD and DVD drives),magnetic disk drives, magnetic tape drives (including LTO, DLT, DAT andDCC), flash drives, program cartridges and cartridge interfaces such asthose found in video game devices, removable memory chips such as EPROMor PROM, emerging storage media, such as holographic storage, or similarstorage media as known in the art. This additional memory 514 may bephysically internal to the computer 506, or external as shown in FIG. 5,or both.

The computer system 500 may also include other similar means forallowing computer programs or other instructions to be loaded. Suchmeans can include, for example, a communications interface 516 whichallows software and data to be transferred between the computer system500 and external systems and networks. Examples of communicationsinterface 516 can include a modem, a network interface such as anEthernet card, a wireless communication interface, or a serial orparallel communications port. Software and data transferred viacommunications interface 516 are in the form of signals which can beelectronic, acoustic, electromagnetic, optical or other signals capableof being received by communications interface 516. Multiple interfaces,of course, can be provided on a single computer system 500.

Input and output to and from the computer 506 may be administered by theinput/output (I/O) interface 518. This I/O interface 518 administerscontrol of the display 502, keyboard 504A, external devices 508 and/orother such components of the computer system 500. The computer 506 mayalso include a graphical processing unit (GPU) 520. The latter may alsobe used for computational purposes as an adjunct to, or instead of, the(CPU) 510, for mathematical calculations. The various components of thecomputer system 500 are coupled to one another either directly or bycoupling to suitable buses.

The above-described embodiments are intended to be examples only. Thepresent disclosures may be embodied in other specific forms withoutdeparting from their spirit or essential characteristics. The describedexamples are to be considered in all respects only as illustrative, notrestrictive. The scope of the invention is, therefore, indicated by theappended claims, rather than by the foregoing description. All changeswithin the meaning and range of equivalency of the claims are to beembraced within their scope. The scope of the claims should not belimited by the preferred embodiments set forth in the examples, butshould be given the broadest interpretation consistent with thedescription as a whole.

1. A computer-implemented method for identifying alternate routing pathsfor network traffic between a known first network and a destinationwithin an unknown second network on the Internet wherein: an initialrouting path for the network traffic between the first network and thedestination within the unknown second network crosses at least oneinterconnecting transit network; the method comprising: a. obtaining, ata processor of a computer, destination IP address information for thedestination within the unknown second network; b. the processorcomparing the destination IP address information to an Internet databaseto determine an identity of the second network; and c. the processorusing the identity of the second network to identify a second routingpath comprising a peer connection between the first network and thesecond network wherein the peer connection and the second routing pathdiffers from the initial routing path and bypasses the at least oneinterconnecting transit network.